2.1.1 Summary stats checks

reviews2.text[, c('incentivized','sentiment_mean')] %>%
  group_by(incentivized) %>%
  summarize_all(mean, na.rm = TRUE)

## # A tibble: 2 x 2
##   incentivized     sentiment_mean
##   <fct>                     <dbl>
## 1 non-incentivized          0.293
## 2 incentivized              0.254

Note: The incentivized sentiment mean is lower, suprisingly. However, the ‘sentimentr’ package is not perfect either.

For example, here is an incentivized review with negative mean sentiment score, yet the content is relatively positive.

knitr::kable(reviews_sentences[reviews_sentences$recid=='100125154',c('text','sentence_sentiment')],
             caption = "An Example of Incentivized Review with Positive Content but Negative Sentiment Score", floating.environment="sidewaystable")

As we see, there’s quite a variation in the sentence sentiment in this particular case. Would this be an example for further analysis on the extreme variance within each review?

But it’s also foreseeable that the variation within non-incentivized reviews would be less prounounced since they’re a lot shorter.

2.1.2 Plots

Boxplot: rating vs. sentence_sentiment

2.1.3 Fixed Effect Linear Model

## fix effect linear model
## Use sentence sentiment score to replce rating
formula.fe <- sentiment_mean ~ incentivized + is_deleted + verified_purchaser
model.fe <- plm(data = reviews2.text, formula = formula.fe, index = c('item_id'), model = 'within')
# get the model summary
summary(model.fe)

## Oneway (individual) effect Within Model
## 
## Call:
## plm(formula = formula.fe, data = reviews2.text, model = "within", 
##     index = c("item_id"))
## 
## Unbalanced Panel: n = 101, T = 29-10134, N = 264016
## 
## Residuals:
##     Min.  1st Qu.   Median  3rd Qu.     Max. 
## -3.29888 -0.19515 -0.02265  0.17020  2.69732 
## 
## Coefficients:
##                              Estimate Std. Error t-value  Pr(>|t|)    
## incentivizedincentivized   -0.0172033  0.0076676 -2.2436   0.02486 *  
## is_deleteddeleted           0.0201613  0.0025538  7.8946 2.923e-15 ***
## verified_purchaserverified  0.0190881  0.0025959  7.3531 1.943e-13 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Total Sum of Squares:    24635
## Residual Sum of Squares: 24624
## R-Squared:      0.00043385
## Adj. R-Squared: 4.3737e-05
## F-statistic: 38.1826 on 3 and 263912 DF, p-value: < 2.22e-16

2.1.4 Correlation: rating vs. review sentiment

cor.test(reviews2.text$rating, reviews2.text$sentiment_mean, method=c("pearson", "kendall", "spearman"))

## 
##  Pearson's product-moment correlation
## 
## data:  reviews2.text$rating and reviews2.text$sentiment_mean
## t = 297.47, df = 264014, p-value < 2.2e-16
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.4981655 0.5038793
## sample estimates:
##       cor 
## 0.5010279

2.2.1 Summary stats checks

reviews2.text[, c('incentivized','sentiment_sum')] %>%
  group_by(incentivized) %>%
  summarize_all(mean, na.rm = TRUE)

## # A tibble: 2 x 2
##   incentivized     sentiment_sum
##   <fct>                    <dbl>
## 1 non-incentivized         0.614
## 2 incentivized             2.47

Note: The incentivized review sentiment sum is a lot higher than the non-incentivized group. Going back to the suprising finding in section 2.1.1, combined with our previous finding that incentivized reviews could be a lot longer than non-incentivized ones, the higher sum could be explained.

2.2.2 Plots

Boxplot: rating vs. sentence_sentiment

2.2.3 Fixed Effect Linear Model

## fix effect linear model
## Use sentence sentiment score to replce rating
formula.fe <- sentiment_sum ~ incentivized + is_deleted + verified_purchaser
model.fe <- plm(data = reviews2.text, formula = formula.fe, index = c('item_id'), model = 'within')
# get the model summary
summary(model.fe)

## Oneway (individual) effect Within Model
## 
## Call:
## plm(formula = formula.fe, data = reviews2.text, model = "within", 
##     index = c("item_id"))
## 
## Unbalanced Panel: n = 101, T = 29-10134, N = 264016
## 
## Residuals:
##     Min.  1st Qu.   Median  3rd Qu.     Max. 
## -4.66658 -0.41532 -0.06511  0.34090 11.80856 
## 
## Coefficients:
##                              Estimate Std. Error t-value  Pr(>|t|)    
## incentivizedincentivized    1.6215649  0.0175074  92.622 < 2.2e-16 ***
## is_deleteddeleted           0.1170905  0.0058311  20.081 < 2.2e-16 ***
## verified_purchaserverified -0.1185700  0.0059273 -20.004 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Total Sum of Squares:    134740
## Residual Sum of Squares: 128380
## R-Squared:      0.047215
## Adj. R-Squared: 0.046843
## F-statistic: 4359.33 on 3 and 263912 DF, p-value: < 2.22e-16

2.2.4 Correlation: rating vs. review sentiment

cor.test(reviews2.text$rating, reviews2.text$sentiment_sum, method=c("pearson", "kendall", "spearman"))

## 
##  Pearson's product-moment correlation
## 
## data:  reviews2.text$rating and reviews2.text$sentiment_sum
## t = 253.22, df = 264014, p-value < 2.2e-16
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.4389741 0.4451123
## sample estimates:
##       cor 
## 0.4420484

Although it’s still significant, the mean sentiment score shows a stronger correlation with ratings.